A Generative Framework for Real Time Object Detection

Recent imaging studies show that the human brain has structures specialized for the detection of eyes and the recognition of eyerelated behavior. There is some evidence that such systems may be innate and play an important role in infant social development. The development of machine perception systems that detect eyes and analyze eye behavior will also enable new approaches to humancomputer interaction that emphasize natural, face-to-face communication with the user. For such systems to have an impact in everyday life it is important for them to work robustly in natural, unconstrained conditions. We formulate a probabilistic model of image generation and derive optimal inference algorithms for finding eyes within this framework. The approach models the image as a collage of patches of arbitrary size, some of which contain the object of interest and some of which are background. The approach requires development of likelihood-ratio models for object versus background generated patches. These models are learned using boosting methods. One advantage of the generative approach proposed here is that it makes explicit the conditions under which the approach is optimal, thus facilitating progress towards methods that model the image generation process in more realistic ways. The approach proposed here searches the entire image plane in each frame, making it resistant to fast, unpredictable motion. The system is robust to changes in lighting, illumination, and differences in facial structure, including facial expressions and eyeglasses. Furthermore, the system can simultaneously track the eyes and blinks of multiple individuals. We also present pilot results using this system for analysis of eye-openness in EEG studies. Finally we reflect on how the development of perceptive systems like this may help advance our understanding of the human brain.